Lenticular-based autostereoscopic displays, such as StereoGraphics' SynthaGram® display, present the user with a realistic stereoscopic visual scene representation by displaying multiple perspective views. The user's eyes see a pair of images from the array of perspective views and a stereoscopic image is formed in the user's mind.
The SynthaGram® display utilizes a lenticular lens array, which is precisely positioned on top of a flat panel digital color display (e.g., LCD and plasma) and is thus used to optically separate the views, which have been combined in the displayed image. This combination process, known as interdigitation or Interzigging™, is a precise mapping of input perspective views to output subpixel components and takes into account the physical properties of the lenticular array and its alignment with the display. The input perspective views are assumed to be created in a digital format, properly composed, aligned, balanced, have appropriate parallax, and each can be of the same horizontal and vertical pixel dimensions. These perspective views can be created by numerous techniques including photography and computer graphics software.
The art of the SynthaGram® display is covered in U.S. Pat. No. 6,519,0888 entitled “Method and Apparatus for Maximizing the Viewing Zone of a Lenticular Stereogram,” and U.S. Pat. No. 6,366,281 entitled “Synthetic Panoramagram.” The art of the SynthaGram® display is also covered in U.S. Publication No. 20020036825 entitled “Autostereoscopic Screen with Greater Clarity,” and U.S. Publication No. 20020011969 entitled “Autostereoscopic Pixel Arrangement Techniques.”
Display pixels are composed of a set of adjacent subpixels, typically organized in a red-blue-green pattern. We will assume in this disclosure that the subpixels are truly adjacent, however the reality is that a small “grout” boundary surrounds each subpixel. The interdigitation mapping describes the correct assignment of subpixels in the perspective views to subpixel elements in the display. The mapping is uniquely based on the size and physical properties of the lenticular lens sheet. Such properties include the pitch of the lenticulars on the sheet, and the slant angle of the lenticulars relative to the flat panel color display. The interdigitation mapping assumes that the display resolution has a fixed pixel dimension (e.g. 1600×1200 pixels). Note that the pixel display format (e.g. 32 bits per pixel or 16 bits per pixel) is not a factor in creating the interdigitation mapping.
In composing an autostereo image, it is common (although not necessary), for the individual perspective views to be of smaller resolution than that of the final displayed image. In our embodiment we utilize 9 perspective views, but it will be clear to those skilled in the art that there is no loss in generality if more or less views are used. We have found when using nine views in our SynthaGram® system, the individual perspective views can be ⅓ the horizontal and vertical resolution of the final displayed autostereo image without any loss of detail.
It is also important to note that the aspect ratio of the individual perspective views is not required to be the same as the displayed image. Well known scaling and centering operations can be used to properly compose and present an interdigitated image in such cases.
Computing the interdigitation mapping is an essential step in computer software based autostereo composition programs. Although the interdigitation mapping relates perspective view subpixel locations to display subpixel locations, this mapping can be simplified. The simplification is based on realizing that the locations in the views and the display pixel locations are proportional to one another. Hence, simply determining which view is mapped to each display subpixel location will suffice for defining the interdigitation mapping. Therefore, once we can determine the correct view to map to a subpixel in the final displayed image, the actual color value to use in the view is determined by selecting the color subpixel at the proportional location in the displayed image.
The computation of interdigitation mapping is disclosed in U.S. Publication No. 20020011969, and is a process we call INTERZIGGING™. In this disclosure, we often use the generic term interdigitation because the process we describe is applicable to interdigitation procedures in general, as well as our proprietary INTERZIG™ process. The length of time to calculate this map is directly proportional to the resolution of the autostereo display. The mapping is one-to-one and uniquely defines the intermixing of perspective views to the subpixels in the displayed image. Stereoscopic properties such as the zero-parallax location in the viewed scene and parallax strength are already inherently expressed in the perspective images, which are considered input to the process and are not affected by the interdigitation mapping.
Our research with the SynthaGram® technology has led us to the conclusion that applying a special filter by a convolution process can produce an improved, sharper effect. Experiments have determined that this filter is best applied as a post process to the interdigitated image in a manner similar to any other filtering process.
There are various categories of computer applications that can be designed for use with such an autostereoscopic display. Such categories can include still image viewing, movie viewing, and real-time interactive animation or simulation. Although windowed computer applications are also supported on autostereoscopic displays, our discussion here is focused on full screen applications. It should be obvious to those skilled in the art that the hardware-based implementations we discuss below are readily adaptable to windowed-based computer applications.
Given the high computational cost of creating the interdigitation mapping, it is advantageous to minimize the number of times the mapping is calculated. Thus, rather than calculating the mapping on each video frame or animation update, the mapping can be computed directly in memory or read from disk file at program initialization, once the desired output monitor and number of views is selected or known.
The trade off between reading the mapping from a disk file or calculating “on-the-fly” is a software application design consideration: increase initialization time versus larger disk space requirements for the application. In either case, the application constructs the interdigitation mapping in computer memory for later use.
The naive software-only approach for using the interdigitation mapping, without leveraging the power of graphic hardware, would be to build an output autostereo image display by iterating through each output subpixel component, selecting the perspective view as indicated by the interdigitation mapping, and then determining the subpixel color value by a proportional lookup in the selected perspective view. Although this lookup approach is computationally expensive, it may be totally acceptable for an image viewer application, which does not require a high output frame rate.
A more efficient approach is to leverage the graphics and image processing capabilities found in the graphics processing units (GPU's) found on state-of-the-art graphics hardware. These graphics cards contain sufficient memory to hold textures and geometry, do lighting and texture operations, and implement the standard graphics transformations in a pipeline approach. These cards are also are programmable, allowing for assembler language-like programs to run and perform calculations per geometric vertex (vertex shaders) and per pixel (pixel shaders). High-level graphics languages such as DirectX and OpenGL have been enhanced to provide application support for these graphic card features. New higher level languages such as Cg and HLSL also aid in giving software implementers the opportunity to migrate complex algorithms such as interdigitation to the graphics card level, without the need for writing code in a low level assembler-like language.
A special feature on these cards is the texture pipeline. In 3D graphics, textures are 1D, 2D or 3D arrays of color, which are essential in providing realistic coloring effects to geometric polygons (typically triangles). Pixels in these textures are called Texels. Textures have their own coordinate system, usually referenced by a uv coordinate system, to which polygons are mapped. This texture mapping is utilized as follows: Each vertex in the polygon is assigned a texture uv coordinate. The graphics engine can then interpolate the color over the entire polygon. A texture pipeline is a hardware implementation that allows textures to be combined in stages, building up a desired special effect. These stages are applied very efficiently to geometry in a single pass of the geometry. Specifically this is called either multiple texture blending or single-pass multitexturing. The present invention takes advantage of these built-in features to provide an effective interdigitation process.